Search Engine

A search engine is a special Web program that searches the contents of a database of available Web pages and other resources. Search engines provide information related to specific topics or keywords supplied by a user.

URLs:

Excite

Research Online International's free search engine uses fuzzy logic to match related sites to specific search terms, and includes NetReviews (in-depth online information) and a Bulletin of regularly updated topical reports.

Yahoo!

Yahoo is one of the most popular search engines on the Internet, and rightly so, because of the wide range of resources and capabilities that make it worthwhile consulting.

Lycos

Lycos is a search engine developed at Carnegie-Mellon University, one of the premier computer science research and teaching institutions in the world. Now set up as spin-off Lycos, Inc., this group delivers one of the longest-running and furthest reaching of all the Web-based search engines.

WebCrawler

The WebCrawler Project began as Brian Pinkerton's research project at the Department of Computer Science and Engineering at the University of Washington in Seattle. The project has since moved to America Online, where it continues to operate as a freely available Internet search tool.

Open Text

The guys at Open Text helped to develop the search engine that Yahoo uses; they have their own public search engine, too. Open Text markets products for full-text indexing, search, retrieval, and display. The software offers blazing search and retrieval performance against very large databases, and includes a set of flexible, easy-to-use programming interfaces (APIs).

InfoSeek

Available in both for-a-fee and free varieties, InfoSeek's search engine has garnered a reputation as the best and most powerful Web search engine today. For a mere $9.95 a month, you get 100 queries in this nonpareil of search engines; on the "free" Internet, we thought this was steep until we tried it out. Our final judgment: Worth every penny (but then, we do research for a living)!

W3E References:

Robot

Spider

Wanderer

Worm

Mastering Web Search Techniques

Detail:

Researching resources on the Web is much easier if you use the right tool rather than following random hyperlinks all over Webspace. Search engines represent a class of software tools that can examine and catalog huge amounts of information in helping you locate Web sites of potential interest to your searching needs. This is how most of them work:

• Somewhere in the background, laboring in obscurity, you'll find automated Web-traversing programs called robots, spiders, wanderers, or worms. These programs do nothing but follow hyperlinks wherever they may lead around the Web. Each time they get to a new Web document, they examine and catalog its contents, saving the information for later transmission to a search engine database elsewhere on the Web.
• At regular intervals these automated information gatherers transmit their recent acquisitions to a remote database, where the information is sifted, categorized, checked, and stored for easy access.
• When users use a search engine, they're actually searching a database that has been created and compiled by the efforts of these data-gathering spiders and robots. These searches are handled by a full-blown database management system that communicates with a CGI program that drives the search engine's user input form, to solicit input and report results.
• Using those keywords or search terms that users provide, the database locates "hits" (exact matches) and also "near-hits" (matches with less than a 100% match of the full set of terms supplied). The best search engines employ something called "fuzzy logic" that allows the engine to systematically look for near hits, as well as outright matches. (This capability makes Excite and InfoSeek both so powerful and useful).
• The database management system returns hits and near-hits to the CGI program, where they're transformed into Web pages that deliver the results of your search request. This latter activity is an excellent example of generating HTML on-the-fly, because these pages are put together only after the results are returned.